Musical timbre: bridging perception with semantics

Musical timbre is a complex and multidimensional entity which provides information regarding the properties of a sound source (size, material, etc.). When it comes to music, however, timbre does not merely carry environmental information, but it also conveys aesthetic meaning. In this sense, semantic description of musical tones is used to express perceptual concepts related to artistic intention. Recent advances in sound processing and synthesis technology have enabled the production of unique timbral qualities which cannot be easily associated with a familiar musical instrument. Therefore, verbal description of these qualities facilitates communication between musicians, composers, producers, audio engineers etc. The development of a common semantic framework for musical timbre description could be exploited by intuitive sound synthesis and processing systems and could even influence the way in which music is being consumed. This work investigates the relationship between musical timbre perception and its semantics. A set of listening experiments in which participants from two different language groups (Greek and English) rated isolated musical tones on semantic scales has tested semantic universality of musical timbre. The results suggested that the salient semantic dimensions of timbre, namely: luminance, texture and mass, are indeed largely common between these two languages. The relationship between semantics and perception was further examined by comparing the previously identified semantic space with a perceptual timbre space (resulting from pairwise dissimilarity rating of the same stimuli). The two spaces featured a substantial amount of common variance suggesting that semantic description can largely capture timbre perception. Additionally, the acoustic correlates of the semantic and perceptual dimensions were investigated. This work concludes by introducing the concept of partial timbre through a listening experiment that demonstrates the influence of background white noise on the perception of musical tones. The results show that timbre is a relative percept which is influenced by the auditory environment

An Argument based Creative Assistant for Harmonic Blending

Conceptual blending is a powerful tool for computational creativity where, for example, the properties of two harmonic spaces may be combined in a consistent manner to produce a novel harmonic space. However, deciding about the importance of property features in the input spaces and evaluating the results of conceptual blending is a nontrivial task. In the specific case of musical harmony, defining the salient features of chord transitions and evaluating invented harmonic spaces requires deep musicological background knowledge. In this paper, we propose a creative tool that helps musicologists to evaluate and to enhance harmonic innovation. This tool allows a music expert to specify arguments over given transition properties. These arguments are then considered by the system when defining combinations of features in an idiom-blending process. A music expert can assess whether the new harmonic idiom makes musicological sense and re-adjust the arguments (selection of features) to explore alternative blends that can potentially produce better harmonic spaces. We conclude with a discussion of future work that would further automate the harmonisation process.Comment: 8 pp; submitted to 7th International Conference on Computational Creativit

Interaction between time-varying tone inharmonicity, fundamental frequency and spectral shape affects felt tension and timbral semantics

Musically induced tension has been the subject of thorough study in the music cognition literature but its relationship with timbre is still poorly investigated. This study examines how the dynamic variation of a tone’s inharmonicity may affect a number of auditory qualities, namely brightness, roughness and mass along with felt tension under different acoustical conditions (i.e., F0, spectral shape and type of inharmonicity). Fifty-six musically trained participants gave real-time continuous ratings on eight time-varying stimuli upon the aforementioned qualities. Static ratings over the initial purely harmonic parts of the stimuli were also obtained by a subgroup of the listening panel. The fundamental frequency exhibits the strongest influence on the responses of the four qualities, followed by the type of inharmonicity and the spectral shape to a lesser degree. The profile patterns of mass and brightness proved to be strong predictors for tension profile patterns while the roughness profile magnitudes show a significant main effect on the magnitude of tension profiles. Overall, these results demonstrate that time-varying inharmonicity affects continuous responses on both timbral semantics and tension, while indicating that felt tension may be influenced by underlying timbral qualities in a dynamic context

Interaction between time-varying tone inharmonicity, fundamental frequency and spectral shape affects felt tension and timbral semantics

Musically induced tension has been the subject of thorough study in the music cognition literature but its relationship with timbre is still poorly investigated. This study examines how the dynamic variation of a tone’s inharmonicity may affect a number of auditory qualities, namely brightness, roughness and mass along with felt tension under different acoustical conditions (i.e., F0, spectral shape and type of inharmonicity). Fifty-six musically trained participants gave real-time continuous ratings on eight time-varying stimuli upon the aforementioned qualities. Static ratings over the initial purely harmonic parts of the stimuli were also obtained by a subgroup of the listening panel. The fundamental frequency exhibits the strongest influence on the responses of the four qualities, followed by the type of inharmonicity and the spectral shape to a lesser degree. The profile patterns of mass and brightness proved to be strong predictors for tension profile patterns while the roughness profile magnitudes show a significant main effect on the magnitude of tension profiles. Overall, these results demonstrate that time-varying inharmonicity affects continuous responses on both timbral semantics and tension, while indicating that felt tension may be influenced by underlying timbral qualities in a dynamic context

Modelling Cadence Perception Via Musical Parameter Tuning to Perceptual Data

Part 10: Mining Humanistic Data Workshop (MHDW)International audienceConceptual blending when used as a creative tool combines the features of two input spaces, generating new blended spaces that share the common structure of the inputs, as well as different combinations of their non-common parts. In the case of music, conceptual blending has been employed creatively, among others, in generating new cadences (pairs of chords that conclude musical phrases). Given a specific set of input cadences together with their blends, this paper addresses the following question: are some musical features of cadences more salient than others in defining perceived relations between input and blended cadences? To this end, behavioural data from a pairwise dissimilarity listening test using input and blended cadences as stimuli were collected, thus allowing the construction of a ‘ground-truth’ human-based perceptual space of cadences. Afterwards, the salience of each cadence feature was adjusted through the Differential Evolution (DE) algorithm, providing a system-perceived space of cadences that optimally matched the ground-truth space. Results in a specific example of cadence blending indicated that some features were distinguishably more salient than others. This pilot study was a first step towards building self-aware blending systems and revealed that the salience of features in conceptual blending is an essential part for producing perceptually relevant blends

Learning and Blending Harmonies in the Context of a Melodic Harmonisation Assistant

Part 10: Mining Humanistic Data Workshop (MHDW)International audienceHow can harmony in diverse idioms be represented in a machine learning system and how can learned harmonic descriptions of two musical idioms be blended to create new ones? This paper presents a creative melodic harmonisation assistant that employs statistical learning to learn harmonies from human annotated data in practically any style, blends the harmonies of two user-selected idioms and harmonises user-input melodies. To this end, the category theory algorithmic framework for conceptual blending is utilised for blending chord transition of the input idioms, to generate an extended harmonic idiom that incorporates a creative combination of the two input ones with additional harmonic material. The results indicate that by learning from the annotated data, the presented harmoniser is able to express the harmonic character of diverse idioms in a creative manner, while the blended harmonies extrapolate the two input idioms, creating novel harmonic concepts